Evaluating applications performance in a multi-networked cluster

Traditionally, a cluster is defined as a collection of homogeneous nodes interconnected by a single high performance communication technology. However, in some cases, cluster nodes may be organized into several partitions – subclusters – internally interconnected by one or more selected SAN technologies. In order to constitute a multi-networked cluster, sub-clusters must share a common SAN technology or a bridge facility must be used. In this paper we show how RoCL – a lightweight user-level communication library designed to support multi-threading in a multi-networked environment – manages to exploit such cluster organization. Performance evaluation results obtained by using two partitions of Myrinet and Gigabit SMP nodes demonstrate the usefulness of our approach both for low-level and high-level operation

Scalable multithreading in a low latency myrinet cluster

In this paper we present some implementation details of a programming model – pCoR – that combines primitives to launch remote processes and threads with communication over Myrinet.B asically, we present the efforts we have made to achieve high performance communication among threads of parallel/distributed applications. The expected advantages of multiple threads launched across a low latency cluster of SMP workstations are emphasized with a graphical application that manages huge maps consisting of several JPEG images

ToCL: a thread oriented communication library to interface VIA and GM protocols

In this paper we present ToCL a thread oriented communication library specially designed to fully exploit multithreading in a multi-networked cluster environment. ToCL provides a basic set of primitives to handle zero-copy message passing between application threads spread among cluster nodes. Large messages are fragmented and sent to remote threads as single messages using multiple low-level communication subsystems. The current implementation supports both Myrinet through GM and Gigabit Ethernet through VIA but we plan to extend it to other communication subsystems

Mapping application-level components into hierarchical systems resources

The appropriate organization of application components and their right mapping into physical resources is fundamental to fully exploit cutting edge technologies especially when hierarchical architectures are used. We present a new approach to combine application modelling with resource management. The proposed programming model allows the mixing of message-passing and global memory facilities and integrates them with the high-level abstractions provided for specifying and organizing application components and resources. The methodologies and tools we had designed pave the way to build complex systems through the use of cooperative applications, executed simultaneously, involving multiple users, thus extending the MPMD model

Distributed paged Hash tables

In this paper we present the design and implementation of DPH, a storage layer for cluster environments. DPH is a Distributed Data Structure (DDS) based on the distribution of a paged hash table. It combines main memory with file system resources across the cluster in order to implement a distributed dictionary that can be used for the storage of very large data sets with key based addressing techniques. The DPH storage layer is supported by a collection of cluster-aware utilities and services. Access to the DPH interface is provided by a user-level API. A preliminary performance evaluation shows promising results.Supported by PRODEP III (grant 5.3/N/199.006/00) and SAPIENS (grant 41739/CHS/2001

Orientação ao recurso: um modelo de comunicação para a computação em clusters

A orientação ao recurso é um novo modelo de comunicação e de computação capaz de tirar partido da comunicação de elevado desempenho, no desenvolvimento de uma classe importante de aplicações paralelas/distribuídas de grande escala. Neste contexto, a biblioteca RoCL, especialmente desenhada para a execução em ambientes cluster, de máquinas SMP, usando múltiplos fios-de-execução, permite a exploração de múltiplas tecnologias de comunicação de elevado desempenho e múltiplos protocolos de comunicação, tais como a Myrinet (GM) e a Gigabit Ethernet (VIA). Os recursos são as abstracções usadas para modelar as entidades aplicacionais que podem ser registadas e localizadas através de um serviço básico de directório, distribuído pelos nodos do cluster

Shortcut routing for chord graphs in the domus hash space

We present and evaluate shortcut routing algorithms for Chord graphs in the hash space, specifically developed to accelerate distributed lookups in the Distributed Hash Tables (DHTs) of the Domus framework. The algorithms explore our findings about the relation of exponential and euclidean distances in Chord graphs, in conjunction with the availability, in each DHT node, of multiple routing tables of the underlying Chord graph. The outcome are routing decisions capable of achieving average distances as low as 40% of those offered by Chord’s conventional routing method. Moreover, the supplemental computational effort to take the shortcut routing decisions is sufficiently low to make the algorithms useful in a broad set of application scenarios

High performance multithreaded message passing on a myrinet cluster

The main purpose of this paper is to present the impact of pCoR thread level message passing facilities on applications running in a Myrinet cluster. To exploit Myrinet technology we use the GM library, which provides a limited number of ports as abstractions to name communication end-points. pCoR communication layer multiplexes GM ports by using a dispatcher thread to handle messages to/from a large number of communication entities (working threads). Our approach combines polling operations executed bymultiple threads to avoid unnecessary context switching; a simple mechanism is used to engage working threads into the polling scheme. We aim to reduce the total number of polling operations required to hold message passing system performance

Toward a dynamically balanced cluster oriented DHT

In this paper, we present a model for a cluster oriented Distributed Hash Table (DHT). It introduces software nodes, virtual nodes and partitions as high level entities that, in conjunction with the definition of a certain number of in variants, provide for the balancement of a DHT across a set of heterogeneous cluster nodes. The model has the following major features: a) the share of the hash table handled by each cluster node is a function of its enrollment level in the DHT; b) the enrollment level of a cluster node in the DHT may change dynamically; c) cluster nodes are allowed to dynamically join or leave the DHT. A preliminary evaluation proved that the quality of the balancement of partitions of the hash table across the cluster, measured by the stan dard deviation with relation to the ideal average, surpass the one achieved by using another well known approach.PRODEP III (grant 5.3/N/199.006/00)SAPIENS (grant 41739/CHS/2001

Deploying applications in Multi-SAN SMP clusters

The effective exploitation of multi-SAN SMP clusters and the use of generic clusters to support complex information systems require new approaches. On the one hand, multi-SAN SMP clusters introduce another level of parallelism which is not addressed by conventional programming models that assume a homogeneous cluster. On the other hand, traditional parallel programming environments are mainly used to run scientific computations, using all available resources, and therefore applications made of multiple components, sharing cluster resources or being restricted to a particular cluster partition, are not supported. We present an approach to integrate the representation of physical resources, the modelling of applications and the mapping of application into physical resources. The abstractions we propose allow to combine shared memory, message passing and global memory paradigms